The introduction of high definition television (HDTV) has generated a large amount of publicity because of the superior performance characteristics of HDTV relative to standard definition television (SDTV). HDTV offers approximately twice the veritical and horizontal resolution of standard definition television (SDTV), providing picture quality approaching that of 35 mm film. Since HDTV signals are digital and virtually free of noise, HDTV offers sound quality approaching that of a compact disc (CD) and does not suffer from some of the shortcomings of SDTV such as crosscolor and crosslumina. HDTV also has lower transmission costs because of error correction codes currently only available with digital formats. In addition, HDTV provides multicasting, which enables broadcasters to split programming transmission into multiple channels, and a higher level of interactivity than SDTV, allowing end users to manipulate a television program at the point of reception.
Despite the technical advantages that HDTV provides over SDTV, the widespread implementation and acceptance of HDTV as a new standard depends upon the resolution of several important issues. One of the most important issues is how to implement HDTV while continuing to support the large number of existing SDTV display devices already in use. Most SDTV display devices cannot decode an HDTV signal and cannot be used in a pure HDTV broadcast environment. Moreover, the relatively high cost of HDTV display devices makes replacing existing SDTV display devices with HDTV display devices impractical. Thus, a widespread implementation of HDTV must address the problem of how to support the existing SDTV display devices currently in use.
One solution to this problem is for service providers to transmit programs in both HDTV and SDTV formats (simulcast). This could be done by transmitting both an HDTV signal and an SDTV signal, or by augmenting an SDTV signal with additional information necessary to "enhance" the SDTV signal to provide a HDTV signal. With this approach, SDTV display devices only process the SDTV portion of the signal while ignoring the additional information. On the other hand, HDTV display devices process both the SDTV signal information and the additional information. Although this approach solves the problem of how to implement HDTV while supporting existing SDTV display devices, it consumes a considerable portion of the available distribution bandwidth making it very undesirable to service providers.
An alternative solution is for service providers to transmit programs in HDTV format only and for all display units to incorporate a special decoder, sometimes referred to as an "all format decoder," that has the ability to decode both standard definition (SD) and high definition (HD) video from the HDTV signal. This approach is attractive from several standpoints. First, an all format decoder can be included in all new HDTV display devices. In addition, an all format decoder can be sold separately as a stand-alone unit that can be attached to existing SDTV display devices. This allows new HDTV display devices can take full advantage of the new HDTV technology while allowing existing SDTV display devices to continue to be used.
Conventional all-format decoders consist of an SDTV decoder, e.g. an MPEG-2 decoder, with a number of enhancements. An HDTV signal is first preprocessed to discard or decimate information that is not needed for a standard definition display, such as high-frequency coefficients. This reduces the amount of data in the preprocessed HDTV signal, reducing the required bandwidth and allowing a smaller channel buffer to be used. The reduced amount of data also simplifies subsequent processing of the preprocessed signal.
Next, the remaining portions of the encoded image data are decoded using any of the well known video sampling techniques such as a high definition 4:4:4 video sampling technique, or a lower definition video sampling technique such as 4:2:2 or 4:2:0. Conventional MPEG decoders use an inverse discrete cosine transform (IDCT) process to decode video-related data that was previously encoded using a discrete cosine transform (DUCT) process. Digital Video: An Introduction To MPEG-2 by Barry G. Haskell, Atul Puri and Arun N. Netravali, provides useful background information on decoding video-related data and is incorporated by reference herein in its entirety.
Image data encoded and decoded by conventional encoders and decoders typically includes luminance data (Y.sub.C) and two types of chrominance data referred to as (U.sub.C) and (V.sub.C). To display a high definition image (1920 by 1080 pixels), a typical decoder provides 1920 by 1080 pixels of luminance-related data and 960 by 540 pixels of chrominance-related data. The resulting data provides a 4:2:0 image having 1920 by 1080 pixels.
Following the decoding process, the decoded image data is then filtered and/or decimated to further reduce the image for display on a lower resolution display. By way of example, the lower definition image provided by certain decoders is 1/4 the size (1/4 number of pixels) of the original higher definition image. Thus, for a 1920 by 1080 pixel image (2,073,600 total pixels), the lower definition image is 960 by 540 pixels (518,400 pixels). Thus, a lower definition image has less information than a higher definition image.
Consider the conventional system 100 illustrated in FIG. 1 for encoding, transmitting and decoding image data. An image is scanned or otherwise processed to create image data 102, which is typically in the form of an image file. Image data 102 is provided to an encoder 104. Encoder 104 encodes image data 102 and provides encoded image data The encoded image data is transmitted to a decoder 106 via a link 108. Link 108 can include one or more communication mediums and/or systems and supporting apparatuses that are configured to carry the encoded image data between encoder 104 and decoder 106. Examples of link 108 include, but are not limited to, a telephone system, a cable television system, a broadcast television system (direct or indirect), a satellite broadcast system (direct or indirect), one or more computer networks and/or buses, the Internet, an intranet, and any software, hardware and other communication systems and equipment associated therewith for transmitting encoded data between two locations.
Decoder 106 decodes the encoded image data received from encoder 104 and provides decoded image data 110 that is suitable for reproduction with a display 112. Ideally, decoded image data 110 is identical to image data 102. However, limitations in the encoding and decoding processes performed by encoder 104 and decoder 106 respectively, sometimes alter the data, adversely affecting the quality of decoded image data 110 when displayed on display 112.
In certain embodiments, encoder 104 and/or decoder 106 include one or more processors that each are coupled to a memory. The processor(s) respond to computer implemented instructions stored within the memories to encode or decode image data 102 as required. In other embodiments, encoder 104 and/or decoder 106 include logic that is configured to encode or decode image data 102 as required.
FIG. 2A is a block diagram depicting a conventional system 200 for reducing a high definition image to a lower definition image. Image data 202 is provided to an encoder 204 that encodes image data 202 and provides encoded image data. Encoder 204 includes a DCT process 206 that encodes image data 202 using a DCT.
The encoded image data provided by encoder 204 is transmitted to a decoder 208 via a link 210. Link 210 communicatively couples encoder 204 and decoder 208 and provides for the transmission of data between encoder 204 and decoder 208 as previously described in regard to link 108 of FIG. 1.
Decoder 208 includes a pre-parser process 212, an IDCT process 214 and a post filter process 216. Pre-parser process 212 decimates, filters, masks, and/or otherwise reduces the amount of encoded image data from encoder 204 and provides a subset of the encoded image data to IDCT process 214. IDCT process 214 decodes the subset of the encoded image data and provides a decoded subset of the encoded image data to post filter process 216.
Post filter process 216 further processes and configures the decoded subset of the encoded image data, typically by decimating, filtering and/or otherwise down-sampling the decoded subset of the encoded image data and provides reduced image data 218. Reduced image data 218 represents a lower definition image than the image represented by decoded image data 110 (FIG. 1) and is suitable for display on a lower resolution display.
FIG. 2B depicts example matrix operations 220 associated with a conventional DCT/IDCT process. Matrix D is an 8 by 8 matrix (e.g., a macroblock) of image data that is multiplied by the 8 by 8 DCT/IDCT coefficient matrixes C and C.sup.T to produce an 8 by 8 matrix T of image data. The coefficients applied in the matrices of FIG. 2B are based upon a conventional DCT such as: ##EQU1##
with:
u, v, x, y=0, 1, 2, . . . N-1 PA1 x, y are spatial coordinates in the sample domain and PA1 u, v are coordinates in the transform domain ##EQU2##
where:
and a conventional IDCT such as: ##EQU3##
This conventional DCT and conventional IDCT can be found in Annex A of the Revised Text for ITU-T Recommendation H.262/ISO/IEC 13818-2:1995, Information Technology--Generic Coding of Moving Pictures and Associated Audio Information, which is incorporated herein in its entirety.
Table 1 of this application contains an example computer program that implements the conventional IDCT above. Section T100 of Table 1 and Section T200 of Table 2 illustrate mathematical steps that implement the IDCT. Sections T202 and T204 of Table 2 illustrate coefficients used with the aforementioned DCT and IDCT respectively.
Although the all-format encoder approach has significant advantages over other approaches for implementing HDTV, the conventional approaches for converting an HDTV image to a lower definition image suitable for display on an SDTV display device have some significant limitations. Specifically, conventional approaches cause video-related information to be lost both before and after the IDCT process adversely affecting the symmetry of the resulting decoded image, providing a lower quality image.
Based on the need to convert high definition images to relatively lower definition images and the limitations in the prior approaches, an approach for converting high definition images to relatively lower definition images that avoids the problems associated with the prior approaches is highly desirable.